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FIELD OF THE INVENTION 

The invention relates to user interfaces for computer programs, and more 
particularly to user interfaces in the field of machine vision. 

BACKGROUND OF THE INVENTION 

In the field of factory automation, the high cost of implementing machine 
vision technology represents a significant barrier to the wide-spread adoption of 
industrial image sensors. Traditionally, a large part of this cost has been 
attributed to the expense of software development. Programmers with 
specialized machine vision experience are often required to accomplish 
ostensibly simple machine vision tasks. Although many attempts have been 
made to simplify application development, there has been no clear solution to the 
machine vision ease-of-use problem. 

The superior ease-of-use characteristics of electronic spreadsheets are well 
known, especially in such fields as office automation. The use of electronic 
spreadsheets in the field of machine vision is also known, although in the past, 
the spreadsheet metaphor could not be adequately exploitedlor such 
applications. One reason for this is that machine vision programming tasks 
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typically involve the manipulation of images and other large data sets, a task 
poorly suited to conventional electronic spreadsheets. 

A standard spreadsheet consists of a rectangular grid of cells, each cell 
representing a single value that is, in turn, a function of some number of other 
cells in the grid. The standard electronic spreadsheet is suited for implementing 
formulas that require a relatively small number of input values and generate a 
single scalar output. In contrast, certain machine vision functions are 
parameterized by arguments representing a million or more pixel values. Since 
there is no facility in a standard electronic spreadsheet that enables a single cell 
to represent multiple values, the standard spreadsheet model is clearly not suited 
to support functions that operate on images and other large data sets. 

In the fields of scientific visualization and computer graphics, the term 
"spreadsheet" has been loosely applied to certain programs that manage multiple 
images and other data representations in a two-dimensional rectangular grid 
within a bounding window. These programs, typically designed for graphical 
editing and array processing, essentially, allow individual data cells to be 
formatted as data arrays. However, in order to accommodate' data sets of varying 

'i 

size, these interfaces generally abandon the strict two-dimensional grid 
relationship between cells that is the essence of the spreadsheet paradigm. In 
fact, most of these programs are unrecognizable as spreadsheets, and are 
generally unsuited for machine vision. 
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SUMMARY OF THE INVENTION 

The invention provides methods for adapting electronic spreadsheets to 
applications in the field of industrial machine vision. The invention also 
substantially preserves the fundamental characteristics of conventional electronic 
spreadsheets. The invention also makes efficient use of a given display area 
and resolution, using for example, a semitransparent spreadsheet superimposed 
over an image of the contents of a data buffer. Further, the invention facilitates 
and simplifies the manipulation of images and other large data sets, while 
preserving the ease-of-use, and other advantages and benefits, of the 
spreadsheet paradigm. 

The invention exploits a new class of object-oriented spreadsheet functions 
that incorporate internal data buffering. In addition, the invention provides 
methods by which the graphical representation of buffered data sets may be 
displayed simultaneously with a spreadsheet grid, preferably' as a graphics layer 
positioned beneath a variably transparent spreadsheet grid. K split-screen and 
dual screen approach can also be employed for displaying the buffered data sets 
alongside the electronic spreadsheet. 

In a preferred embodiment, a conventional spreadsheet is extended to 
include data buffering functions, herein referred to as "single method objects", 
because from the perspective of the spreadsheet programmer, there is only one 
public member function, i.e., only one public method. For exahnple, in accordance 
with the invention, the familiar "search" tool of machine vision is the public 
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member function of the search object, which includes a search function, as well 
as data associated with the search function. 

A single method object can be instantiated and assigned to a spreadsheet 
cell within a variably transparent grid superimposed on a graphical display buffer, 
the contents of the graphical display buffer being determined in accordance with 
the currently selected cell. If an empty cell, or a cell containing a conventional 
spreadsheet formula is selected, the image last acquired is stored and displayed 
in the underlying graphical display buffer. If a cell containing a single method 
object is selected, a graphical representation of the object data members is 
stored in the underlying buffer and displayed. 

The foregoing methods selectively extend data dimensionality to meet the 
practical requirements of machine vision processing, while still retaining the 
fundamental characteristics and ease-of-use of a conventional electronic 
spreadsheet. 

BRIEF DESCRIPTION OF DRAWING U 

The invention will be more fully understood from the following detailed 
description, in conjunction with the following figures, wherein: 

Fig. 1 is a block diagram of a computer system for suitable for practicing the 
invention; 

Fig. 2 illustrates a superposition of graphics layers to obtain the user- 
interface graphics display of the monitor of Fig. 1 ; 
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Fig. 3 is a depiction of the graphical user interface of the invention with cell 
A1 selected of a semi-transparent spread sheet that is superimposed on the 
image corresponding to the single method object instantiated in cell A1 ; 



Fig. 4 is a depiction of the graphical user interface of the invention with cell 
A2 selected of a semi-transparent spread sheet that is superimposed on the 



Fig. 5 is a depiction of the graphical user interface of the invention with cell 
A3 selected of a semi-transparent spread sheet that is superimposed on the 
image corresponding to the single method object instantiated in cell A3; and 

Fig. 6 is a depiction of the graphical user interface of the invention with cell 
A4 selected of a semi-transparent spread sheet that is superimposed on the 
image corresponding to the image last acquired. 

DETAILED DESCRIPTION . i: 

Referring to Fig. 1 , a preferred spreadsheet user interface for machine vision 
applications includes a monitor 1 1 connected by cable to a computer 12. The 
computer 12 includes image acquisition, processing, and graphical display 
capability. A camera and lens assembly 13 has an object surface 15 within its 
field of view, and is connected to the computer 1 2. Also, connected to the 
computer 12 is a user input device, such as a game controller 14, or a standard 
keyboard (not shown). 

The screen of the monitor 1 1 depicts a semitransparent spreadsheet 1 6 
superimposed on an image and graphics layer 17 to form a composite display of 
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image corresponding to the single method object instantiated in cell A2; 




the invention. The transparency of the semitransparent spreadsheet 16 can be 
adjusted by the user. The semitransparent spreadsheet 16 can also be 
displayed in a split-screen arrangement wherein the image and graphics layer 17 
is displayed on one portion of the screen of the monitor 1 1 , and the 
semitransparent spreadsheet 16, which may be adjusted so as to no longer be 
transparent, can be displayed in another portion of the screen of the monitor 1 1 . 
Alternatively, the image and graphics layer 17 can be displayed on the screen of 
the monitor 1 1 , and the semitransparent spreadsheet 1 6, which may be adjusted 
so as to no longer be transparent, can be displayed on the screen of a second 
monitor (not shown). 

Fig. 2 shows the superposition combination of the contents of data display 
buffer 21 and the semitransparent spreadsheet overlay 22 to form a composite 
display 23. Not shown are the split-screen combination, and the dual-screen 
combination of the contents of data display buffer 21 and the semitransparent 
spreadsheet overlay 22. ■ ' 

Figs. 3, 4, 5, and 6 illustrate the effect of spreadsheet cell selection on the 
underlying data display buffer. The contents of the underlying data display buffer 
changes in accordance with the particular cell that is currently selected. Also, the 
contents of the formula line changes in accordance with the particular cell that is 
currently selected. Note that only the data contents of the selected cell is 
displayed; all of the other data associated with the objects underlying the other 
cells remains invisible to the user. By contrast, all of the data in a standard 
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spreadsheet is visible to a user, including a large standard spreadsheet, 
notwithstanding the need to scroll over the large spreadsheet. 

In monitor 31 of Fig. 3, spreadsheet focus (highlight) 33, indicated by the 
frame border (or any selection highlighting), is on cell A1 . Formula line 32 
indicates that cell A1 is assigned the single method object ACQUIRE, which 
contains an image buffer resulting from its image acquisition method. The 
underlying data display buffer 34 shows the content of the image buffer 
associated with ACQUIRE. 

In monitor 41 of Fig. 4, spreadsheet focus 43 is on cell A2. Formula line 42 
indicates that cell A2 is assigned the single method object PROCESS which 
contains the image buffer that results from the processing a rectangular region 
45 in the image associated with cell A1 . The other four arguments of the object 
PROCESS represent the coordinates of the upper-left corner, and the height and 
width, of the rectangular region 45 in the image associated with cell A1 . The 
underlying data display buffer 44 shows the content of the image buffer 
associated with the processed image. 

In monitor 51 of Fig. 5, spreadsheet focus 53 is on cell A3. Formula line 52 
indicates that cell A3 in this example is assigned the single method object 
HISTEXTRACT which contains a histogram array extracted from a rectangular 
region 55 in the processed image assigned to cell A2. The other four arguments 
of the object PROCESS represent the coordinates of the upper-left corner, and 
the height and width, of the rectangular region 55 in the image associated with 
cell A2. The underlying data display 54 shows the content of the input image 
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buffer plus a graphical representation of the histogram array associated with the 
content of the input image buffer. y 

In monitor 61 of Fig. 6, spreadsheet focus 63 is on cell A4. Formula line 62 



the histogram array assigned to cell A3. The underlying data display buffer 64 
shows the content of the acquired image. 

The forgoing illustrates how single method objects co-operate with a variably 
transparent spreadsheet overlaid on the contents of a data display buffer so as to 
efficiently and elegantly extend the conventional spreadsheet paradigm to 
accommodate images and other large data sets. 



Other modifications and implementations will occur to those skilled in the 
art without departing from the spirit and the scope of the invention as claimed. 
Accordingly, the above description is not intended to limit the invention except as 
indicated in the following claims. 



indicates that cell A4 in this example is assigned a contrast value computed from 
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